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1. INTRODUCTION 
Considering the next unconstrained optimization problem, 


min f(x), x E€ R”, (1) 


where f: R” > R is a continuous and differentiable. The conjugate gradient (CG) method considered as one 
of the choicest for solving (1), particularly for the case n is large. The nonlinear conjugate gradient method’s 
iterative formula is given by, 


Xk+1 = Xk + kdk 5 (2) 


where x;, is present iterate point and a; is step length, which is calculated by performing a line search, and dx 
is the search direction, which is defined by, 


Jk+1, if k=0 
dk+1 = (3) 
—Gxr1 t Pryde if k21, 
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where Pp is paramter. The classical conjugate gradient method includes the Hestenes and Stiefel [1]. The 
Fletcher and Reeves [2]. The Polak [3], method Polyak and Ribiere [4]. The conjugate descent method [5]. The 
Liu and Storey method, [6] and the Dai and Yuan method [7], the parameters Pg of these methods are as 
follow: 


HS _ 9k (Ik-Ik-1) FR _ 9k 9k PRP _ 9k(Gk—-9k-1) 
HHS = 9 Ge Bea) BER = fe oe BRR? = Se Gir daw) 
(9k-9k-1)" dk-1 Ik-1 Gk-1 Ik-1 Gk-1 
por =- Tk Ik Bis = _ IkkI) DY — Ik Ik 
k aft Ik-1 k ah Gk’ k (Jk-9k-1)! dk-1 


usually, in the conjugate gradient methods convergence analysis and implementation, the step length a, is 
requireed to satisfy some line search to be imprecise line searches [8]-[11], such as an Armijo line search or a 
strong Wolfe line search. The strong Wolfe line search is utilized to find a, such as: 


fe + andy) — f (Xx) < Sage dr 
lg (Xe + Ody)" del < -o gk dx (4) 


where 6 € (0,=),and o € (6,1). 


The nonlinear conjugate gradient method's sufficient descent condition is as (5). 
9A, < —Cllgxll?, Yk = 1, € (0,1). (5) 


The sufficient decent property and the global convergence have been studied by many researchers such as Baali 
[12] who established the global convergence of the FR method under strong Wolfe line search, Liu et al. [13] 


and Dai and Yuan [14] extended the results to o = . , Gilbert and Nocedal [15], established the global 


convergence property of the PRP* method, the PRP* indicated that is a non-negative parameter. For more 
studies [16]-[18]. 

This paper is organised into four sections. In section 2, a new parameter for the coefficient pẹ is 
proposed followed by an algorithm. The sufficient descent condition and the global convergence analysis 
under strong Wolfe line search is presented in subsection 2.1. In section 3, the numerical performance of the 
new formula versus other well-known conjugate gradient methods are presented. Finally, in section 4, the 
conclusion is presented. 


2. THE MODIFICATION METHOD 
Recently, Rivaie et al. [19], [20], proposed two new formulas as (6), (7). 


RMIL _ Gk (Ik-Ik-1) 6 
By a 2 , ( ) 
lldg-1ll 
T 
RMIL+ — 9k (Ik-Ik-174k-1) 7 
Pk = 2 ? ( ) 
lldg-ll 


Zhang [21] presented an improved formula called BN??? to Wei-Yao-Liu which is given by, 


llgell loron-s| 


gel? 


NPRP — lax-all 
fk = Iga? (8) 
the NPRP method satisfies the descent property given in condition (5). 
Motivated by the in (7) and (8), we propose a modified formula of RMIL+ as: 
2. larll 7 
ligxll lor- 9l if llg |2 > llgrll log | 
BAO = 4 max(lidr-illgk-l IKE Ng eal (9) 
0 , otherwise. 


where AO denotes Abashar and Osman. 
By defining the formula Bf°, we have a new CG method which can be described in Algorithm 2.1. 
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Algorithm 2.1 

Step 0. Initialization, given x) E R”, € > 0,set dọ = —go,k = 0 

Step 1. If ||ggl| < £ , then exit. 

Step 2. Find a, using (4). 

Step 3. Set Xk+1 = XK + dk, Jr+1 = 9X41), if llgnsill < E then stop. 
Step 4. compute fx by the (6, 7, 8, 9 and FR method), generated dx by (3). 
Step 5. Put k = k + 1 and go to Step 2. 


2.1. Analysis of convergence 
In this subsection, the analysis of Algorithm 2.1 is presented. We proved that the algorithm satisfies 
condition (5) and the properties of global convergence. The next lemma is required to simplify the new B/°. 


Lemma. 2.1.1 


A satisfies, BA° LL p >O. 


Siaa Pk 
From the definition (9), we get, 


2__liøxli 
AO = llggll lak- £ lloxll? (10) 
k max(lldg-1l2llgg-1l2) 7 Idea ll? 


lg 1l 


using Cauchy- Schwarz inequality, we get (11). 


lge? — lly loEoal Moel — leel odol 
=- > 


~ max(lidg-1ll2, lgl) © max(lldg-1ll2, lgl 
Be° = 0. (11) 


k 


Lemma. 2.1.2 
Suppose that {g,} and {dp} are generated by the Algorithm 2.1 for o < =, then, 


ily yk > 0. (12) 
lidxll 


Proof. The proof is by induction. For k = 0, l- = 1 < 1, hence (12) holds for k = 0. 


From (3), multiplying by g],, , we get, 


Dhrrdess = -llgk+1 ll? + Pr+19k+1dk 
llgk+ll? = Pr+19k+1dk = Jk+1dk+1 (13) 


from (4) and absolute value of (13), obained, 


llIr+1ll? < lBk+19k+1drl + lgk+1dr+1l 
Geral? < —OBy aM Geell dic] + WGie+ Wiss 


by applying (12), and substitute (10), we get, 


devil? < -0 EÉ IL gelllldll + Nge lllldesl (14) 


xl 
lgs? < -olge ll? t+ Ngee lls 


lIk+ll? < —2øllgk+ll? + lgx llllde+ll 


Grrl? + 2øllgk+ll? < geri lllldr+1ll (15) 


Divided by ||gk+1lllldk+1ll, we get, 


k+l < (16) 


lldk+1ll ~ 1420 


hence this holds true for k + 1. 
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Theorem 2.1.1 
Assume that gx and dp are produced by the methods (2) and (3), respectively, and that the step size 


a, is calculated by (4), if o < > , then the relation, 


—1+20 Ghdy -1-20 
1420 ~ |lgxll2 — 1420’ 


(17) 


holds. Henceforth, condition (5) holds as gg # 0. 
Proof. By induction, true if k = 0, assume (17) is true if k > 0 , from (3), we have: 


Diridees = =llgk+1 ll? + Pr+19k+1dk (18) 


Dividing both sides by ||gg+1ll? 


Jk+1dk+1 b Jk+1dk 
a7 a2 ~ L+ fkin iz 
llgr+ll llgr+ll 


using strong Wolfe condition (4) we have (19). 
lbk+19k+1drl < —0lbkl gk dk 


T T T 
Gk ak Geer dk+1 Ikåk 
-1+0 < <-1-0 19 
Prsi Geil? T geal? Prsa Ideal? (19) 


By using (10) and Cauchy inequality, we have, 


ital? Ig xllll dell < Teer ices ee ee llgk+1ll? Wg xllll dell 
dill? Wgk+all? ~ loess ll? ~ lidel? IlIk+1ll? 


T 
d 
-1 4g lull < nien < 4 — g lal 20) 
ldgll © lgr+ll idyll 


-1+0 


from the induction hypothesis (16), we obtain (21). 


T 
—1+20 d —1-20 
< Jk+1ĉk+1 < 
1+20 llgg+1ll? 1+20 


(21) 


We conclude that (17), holds fork +1. 


Assumption 2.1.1 

(i) The set Q = {x E€ R” f(x) < f (xo)} is bounded with an initial guess x9. 

(ii) f is continuously differentiable and its gradient is Lipschitz continuous in some neighborhood N of Q, 
that is, there exists a constant l > 0 such that ||g (x) — g(y)|| <l|lx-—yll,Vxy E€ N). 


Theorem 2.1.2 
Suppose that Assumption 2.1.1 holds. Let {gx} be obtained by Algorithm 2.1, then jim llggll = 0. 


Proof. We use contradiction, that is, there is a scalar € > 0, such that (22). 


lgxll 2 €, (22) 


From (4), we have (23). 


o 
1+20 


lgidkl < -0 gi-1dk S llgk-1ll?, 


Thus from (3) and (10), we obtain, 


ld ll? < Ilgxll? + 21B lige drl + BE lldx—sIl?, (23) 
Will? < Ilgxll? + ae eal? [Bal + lldr-1ll?, 
2 2 o 2 Moxll? 2 2 
lldillé < Mell + S55 llgx-ll lapai + Pklldg-1ll", 
2 2 o 2 (ll9k-1ll? 
laill? < lgl? + -5 Noell? G5) 


by applying (16), we get, 
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2 
lIdell? < ligil? + lakli? (>) 


1+20 1+20 


oO 
IIdill? < lgl? + lgl? M , M= oya 


ld lI? < (1 + M)llgxll? (24) 


dividing both sides by ||gx||4 to obtain (25). 


arll ~ Maxll? 
dx ||? k (1+M) 

< YE 25 
grlit i=0 | gill? (25) 


dx||? a (1+M) 


Therefor it follows from (22) and (25), 


ldx ll? _— (1+M)k 
llgxll* a 
| 4 


Ikl Sei = o (26) 


lldgl 
llgrl 
lidgll? 


this implies that, 


ye lgxll* = o 
k=0 dy ll? 


this contradicts the condition of Zoutendijk [22]. Hence, the proof is come true. We now state that Algorithm 
2.1 satisfies the property (*). The Property (*) states that: given a method of form (2), (3), and, 


0 <7 < llgkll <7, (27) 
where y and F are positive constant, a method is said to have property. (*), if for all k > 1, there a constant 


b > 1,4 > 0, such that |6,| < b and ||s,|| < A implies |b| < =, where Sp = Xk41 — Xk- 


Lemma 2.1.3 

Let Assumption 2.1.1 be satisfied, then property (*) holds when Algorithm 2.1 applied. 
NE Sui 

Proof. Suppose that (27) holds, set b = yz >1,⁄= FT) > 0. 


From the definition of B/° that, 


løxll løxll 


T a loz gr-1] 2z 
|20| = lox- < lox- < gull -< A =p (28) 
max (ldx—11I7,llox—11l7) llgx-1ll llgk-ll y 


by assumption 2.2.1, and properties of norm, we can get that if Sk-1 <7, then, 


T 
løxl gt loll? Hell og lel Sea 
liggll?- = Ik |9e9k-1| GM (U9 
rae Gel? -pgp lII- lor- lor-l 
k max(lidg-1ll2llIk- l| 7 llgr-1ll2 = llgr-1ll2 i 
T T 
IkIk- IkIk- 
tse onl i ) lgxll (iost noa i ) 
Ik-1 Ik-1 lgrll lgx-9x-11l+llox—1—-9 kl) 
< tis ee 
llgk-1ll? llgg-1ll? lgg-1ll? 
< lgl ge- Ir-alitlgk-Ikal) 2447 1 
= lox = y? 2b 


3. NUMERICAL DISCUSSIONS 

In this section, we ran some experiments to put the Algorithm 2.1 to the test; we referred to test 
problem addressed in Andrei [23]. In order to compare the performance of the proposed formula with those 
of the CG methods listed in (6) and (7). The comparisons were based on the amount of time spent on the 
CPU and the number of iterations. Considered € = 1076 and ||g,|| < £ as a stopping criterion as presented 
in Hillstrom [24]. All test problems in Table | are executed using MATLAB. 


Indonesian J Elec Eng & Comp Sci, Vol. 28, No. 2, November 2022: 1184-1191 


Indonesian J Elec Eng & Comp Sci 


ISSN: 2502-4752 


Table 1. Problem functions list 


m) 


No. Function N Initial point 

1. Six hump 2 (4,4),(10,10) 

2. Booth 2 (25,25),(100,100) 

3. Treccani 2 (5,5),(10,10) 

4. Three hump 2 (-5,5),(-3,12) 

5. Zettl 2 (15,15),(10,10) 

6. Leon 2 (20,20),(50,50) 

ve Matyas 2, (15,15),(6,6) 

8. Wood 4 (6,6,6,6),(13,13,13,13) 

9. Colville 4 (10,10,10,10),(30,30,30,30) 

10. Powell 4 (2,2,2,2),(20,20,20,20) 

11. Power 4 (2,2,2,2),(8,8,8,8) 
10,100 (2,2,...,2),(8,8,...,8) 

12. Extended Peanlty 2,4 (5,5),(11,11),(5,5,5,5),(11,11,11,11) 
100,500 (5353;.25),€11,11,.4.,11) 

13. Generalized Tridiagonal 1 2,4 (5,5),(20,20),(5,5,5,5),(20,20,20,20) 
10,100,500 (5,5,...,5),(20,20,...,20) 

14. Raydan 1 2,4 (3,3),(10,10),(3,3,3,3),(10,10,10,10) 
10,100 (3,3,...,3),(10,10,...,10) 

15. Dixon and Price 4 (80,80,80,80),(150, 150,150,150) 
10,100 (80,80,...,80),(150,150,...,150) 

16. Hager 4 (2,2,2,2),(15,15,15,15) 
10,100 (2,2,+-+;2),(15,15,...,15) 

17. Flether 4 (40,40,40,40),(60,60,60,60) 
10,100,500 (40,40,...,40),(60,60,...,60) 

18. Nonscomp 2 (8,8),(-1,-1) 
10,100 (8,8,...,8),(-1,-1,...,-1) 

19. Extended Freudenstein and Roth 4 (-3,-3,-3,-3),(7,7,7,7) 
10,100,500 (-3,-3,...5-3),(7575---57) 

20. Generalized Tridiagonal 2 4 (1,1,1,1),(4,4,4.4) 
10,100,500 (1,1,...,1),(4,4,...,4) 

21. Extended Quadratic Penalty QP2 4 (6,6,6,6),(14,14,14,14) 
10,100,500, 1000 (6,6,...,6),(14,14,...,14) 

22. Extended Beale 4 (-1,-1,-1,-1),(2,2,2,2) 


23. Diagonal 4 

24. Extended Maratos 

25. Shallow 

26. Extended Rosen Brock 
2T. Extended White and Holst 
28. Quadratic QF2 

29. Extended Denschnb 

30. Extended Himmelblau 


10,100,500, 1000, 10000 
500, 1000, 10000 
10,100,500, 1000 
100,500, 1000, 10000 
100,500, 1000, 10000 
10,100,500, 1000,10000 
10,100,500,1000 
10,100,500, 1000,10000 
10,100,500, 1000,10000 


(-L-1,...5-1),(2,2,-..,2) 
(1,7,...,7),(18,18,...,18) 
(0.5,0.5,...,0.5),(1.5,1.5,... 
(-5,-5,...,-5),(-20,-20,... 
(5,5,...,5),(10,10,...,10) 
(-5,-5,...,-5),(-7,-7,...,-7) 
(2,2,...,2),(60,60,...,60) 
(4,4,....4),(16,16,...,16) 
(30,30,...,30),(200,200,...,200) 
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Figures 1 and 2 display performance results that were established using the performance profile 


proposed by Dolan and More [25]. Based on their performance profile, we take tp s to be the outcome when 
tps 
min{ty,s:s ESF 
Then we can order the values 7, increasingly and plot them versus p,(t), where ps(t) is the ratio 
The order the problem 


the solver s is used to solve problem, and 7, to be the ratio where S is the set of all solvers. 


Totalnümber of problems Clearly the method of top curve is the winner. Overall, a solver with a high p(t) value 


or the curve that seems on top of the Figures is the most effective problem solver. As can be seen in Figures 1 
and 2, our new proposed hast the choicest results whenever it could solve every test problem as in Table 2. 


e el e? e at e5 


t 


Figure 1. Performance results based on the number of iterations 
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aaa ee = D 
ee i 
RMIL 
AO 
------ NPRP 
— —— RMIL+ 
——— FR 
t 
Figure 2. Performance results based on the CPU time 
Table 2. Relative efficiency of the methods 
AO RMIL RMIL+ NPRP FR 
1 0.7812 0.8143 0.8516 0.8094 
CONCLUSION 
In this paper, we presented a parameter for pẹ that has better convergence. Numerical outcomes 
reflected that proposed formula px highlighted better than FR, RMIL, RMIL+ and NPRP. In the future, 


the new formula can be applied under another line search. 
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